{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build a recommender\n",
    "\n",
    "*by [Longqi@Cornell](http://www.cs.cornell.edu/~ylongqi/) licensed under [Creative Commons Attribution 4.0 International License](https://creativecommons.org/licenses/by/4.0/)*\n",
    "\n",
    "This tutorial demonstrates the process of building recommendation algorithms using openrec.tf1. For training and evaluation, please refer to [tutorial-Get Started](https://github.com/ylongqi/openrec/blob/master/tutorials/OpenRec%20Tutorial%20%231.ipynb).\n",
    "\n",
    "Building a recommender takes two steps:\n",
    "* Decompose a recommendation algorithm into **modules** + **a computational graph**.\n",
    "* Inherent from the [Recommender](http://openrec.tf1.readthedocs.io/en/latest/recommenders/openrec.tf1.recommenders.recommender.html) class and overwrite corresponding functions.\n",
    "\n",
    "In this tutorial, we will start by building a simple [PMF (Probablistic Matrix Factorization)](https://papers.nips.cc/paper/3208-probabilistic-matrix-factorization.pdf) recommender and then extend it to a VisualPMF recommender that additionally incorporates visual signals (e.g., leveraging product images for recommendation)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Probablistic Matrix Factorization (PMF)\n",
    "\n",
    "### 1. Decompose PMF into modules + a computational graph\n",
    "PMF can be represented as the following computational graph, and it consists of two reusable components:\n",
    "\n",
    "<img src=\"https://s3.amazonaws.com/cornell-tech-sdl-openrec/files/pmf_flow.png\" width=\"200px\">\n",
    "\n",
    "* Extraction: embeds each user and item id into a vector, and \n",
    "* Interaction: uses dot-product and Mean Square Error (MSE) to model interactions (implicit feedback),\n",
    "\n",
    "### 2. Implementation using OpenRec *Recommender*\n",
    "To implement the modularized PMF, we first map the prior defined components into OpenRec modules, as shown below.\n",
    "\n",
    "<img src=\"https://s3.amazonaws.com/cornell-tech-sdl-openrec/files/openrec_pmf.png\" width=\"200px\">\n",
    "\n",
    "Then we sequentially define inputs, input mappings, extraction modules, and interaction modules as [specified in the Recommender](http://openrec.tf1.readthedocs.io/en/latest/recommenders/openrec.tf1.recommenders.recommender.html). Functions `add_input` and `add_module` should be used to construct inputs/modules. To connect modules and inputs together to form a computational graph, functions `get_input` and `get_module` should be used to retrieve existing inputs/modules.\n",
    "\n",
    "(Note that in the implementation below, we additionally add an **item bias** to capture popularity signals)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class PMF(Recommender):\n",
    "\n",
    "    def __init__(self, batch_size, dim_embed, max_user, max_item,\n",
    "                    test_batch_size=None, l2_reg=None, opt='SGD', sess_config=None):\n",
    "\n",
    "        self._dim_embed = dim_embed\n",
    "\n",
    "        super(PMF, self).__init__(batch_size=batch_size, \n",
    "                                  test_batch_size=test_batch_size,\n",
    "                                  max_user=max_user, \n",
    "                                  max_item=max_item, \n",
    "                                  l2_reg=l2_reg,\n",
    "                                  opt=opt, sess_config=sess_config)\n",
    "\n",
    "    '''\n",
    "    Define inputs using `add_input` function\n",
    "    '''\n",
    "    def _build_user_inputs(self, train=True):\n",
    "        \n",
    "        if train:\n",
    "            self._add_input(name='user_id', dtype='int32', shape=[self._batch_size])\n",
    "        else:\n",
    "            self._add_input(name='user_id', dtype='int32', shape=[None], train=False)\n",
    "    \n",
    "    def _build_item_inputs(self, train=True):\n",
    "        \n",
    "        if train:\n",
    "            self._add_input(name='item_id', dtype='int32', shape=[self._batch_size])\n",
    "        else:\n",
    "            self._add_input(name='item_id', dtype='none', train=False)\n",
    "    \n",
    "    def _build_extra_inputs(self, train=True):\n",
    "        \n",
    "        if train:\n",
    "            self._add_input(name='lables', dtype='float32', shape=[self._batch_size])\n",
    "    \n",
    "    '''\n",
    "    Define input mappings\n",
    "    '''\n",
    "    def _input_mappings(self, batch_data, train):\n",
    "\n",
    "        if train:\n",
    "            return {self._get_input('user_id'): batch_data['user_id_input'],\n",
    "                    self._get_input('item_id'): batch_data['item_id_input'],\n",
    "                    self._get_input('labels'): batch_data['labels']}\n",
    "        else:\n",
    "            return {self._get_input('user_id', train=False): batch_data['user_id_input']}\n",
    "\n",
    "    '''\n",
    "    Define modules using `add_module` function.\n",
    "    '''\n",
    "    def _build_user_extractions(self, train=True):\n",
    "\n",
    "        self._add_module('user_vec', \n",
    "                         LatentFactor(l2_reg=self._l2_reg, init='normal', \n",
    "                                      ids=self._get_input('user_id', train=train),\n",
    "                                      shape=[self._max_user, self._dim_embed], \n",
    "                                      scope='user', reuse=not train), \n",
    "                         train=train)\n",
    "    \n",
    "    def _build_item_extractions(self, train=True):\n",
    "        \n",
    "        self._add_module('item_vec',\n",
    "                         LatentFactor(l2_reg=self._l2_reg, init='normal', \n",
    "                                      ids=self._get_input('item_id', train=train),\n",
    "                                      shape=[self._max_item, self._dim_embed], \n",
    "                                      scope='item', reuse=not train), \n",
    "                         train=train)\n",
    "        self._add_module('item_bias',\n",
    "                         LatentFactor(l2_reg=self._l2_reg, init='zero', \n",
    "                                      ids=self._get_input('item_id', train=train),\n",
    "                                      shape=[self._max_item, 1], \n",
    "                                      scope='item_bias', reuse=not train), \n",
    "                         train=train)\n",
    "\n",
    "    def _build_default_interactions(self, train=True):\n",
    "\n",
    "        self._add_module('interaction',\n",
    "                    PointwiseMSE(user=self._get_module('user_vec', train=train).get_outputs()[0], \n",
    "                                 item=self._get_module('item_vec', train=train).get_outputs()[0],\n",
    "                                 item_bias=self._get_module('item_bias', train=train).get_outputs()[0], \n",
    "                                 labels=self._get_input('labels'), a=1.0, b=1.0, \n",
    "                                 train=train, scope='PointwiseMSE', reuse=not train),\n",
    "                        train=train)\n",
    "\n",
    "    def _build_serving_graph(self):\n",
    "\n",
    "        super(PMF, self)._build_serving_graph()\n",
    "        self._scores = self._get_module('interaction', train=False).get_outputs()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visual Probablistic Matrix Factorization (VisualPMF)\n",
    "\n",
    "Built upon PMF, VisualPMF additionally incorporates visual signals with the following structure (left). The corrsponding OpenRec modules are shown on the right.\n",
    "\n",
    "<img src=\"https://s3.amazonaws.com/cornell-tech-sdl-openrec/files/visual_pmf.png\" width=\"700px\">\n",
    "\n",
    "Because PMF is highly modular, VisualPMF can be impolemented by directly inherenting from PMF and extending functions `build_item_inputs`, `input_mappings`, `build_item_interactions`, and `build_default_fusions`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class VisualPMF(PMF):\n",
    "\n",
    "    def __init__(self, batch_size, max_user, max_item, dim_embed, dims, item_f_source, \n",
    "                 test_batch_size=None, item_serving_size=None, dropout_rate=None,\n",
    "                l2_reg_u=None, l2_reg_mlp=None, l2_reg_v=None, opt='SGD', sess_config=None):\n",
    "\n",
    "        self._dims = dims\n",
    "        self._dropout_rate = dropout_rate\n",
    "        self._item_f_source = item_f_source\n",
    "        self._item_serving_size = item_serving_size\n",
    "\n",
    "        self._l2_reg_u = l2_reg_u\n",
    "        self._l2_reg_mlp = l2_reg_mlp\n",
    "        self._l2_reg_v = l2_reg_v\n",
    "\n",
    "        super(VisualPMF, self).__init__(batch_size=batch_size, \n",
    "                                        max_user=max_user, \n",
    "                                        max_item=max_item, \n",
    "                                        dim_embed=dim_embed,\n",
    "                                        test_batch_size=test_batch_size, \n",
    "                                        opt=opt, sess_config=sess_config)\n",
    "\n",
    "    def _build_item_inputs(self, train=True):\n",
    "        \n",
    "        super(VisualPMF, self)._build_item_inputs(train)\n",
    "        if train:\n",
    "            self._add_input(name='item_vfeature', dtype='float32', \n",
    "                            shape=[self._batch_size, self._item_f_source.shape[1]])\n",
    "        else:\n",
    "            self._add_input(name='item_id', dtype='int32', shape=[None], train=False)\n",
    "            self._add_input(name='item_vfeature', dtype='float32', \n",
    "                            shape=[None, self._item_f_source.shape[1]], train=False)\n",
    "\n",
    "    def _input_mappings(self, batch_data, train):\n",
    "\n",
    "        default_input_map = super(VisualPMF, self)._input_mappings(batch_data=batch_data, \n",
    "                                                                   train=train)\n",
    "        if train:\n",
    "            default_input_map[self._get_input('item_vfeature')] = \\\n",
    "                            self._item_f_source[batch_data['item_id_input']]\n",
    "        else:\n",
    "            default_input_map[self._get_input('item_id', train=False)] = \\\n",
    "                            batch_data['item_id_input']\n",
    "            default_input_map[self._get_input('item_vfeature', train=False)] = \\\n",
    "                            self._item_f_source[batch_data['item_id_input']]\n",
    "        return default_input_map\n",
    "\n",
    "    def _build_item_extractions(self, train=True):\n",
    "\n",
    "        super(VisualPMF, self)._build_item_extractions(train)\n",
    "        self._add_module('item_vf',\n",
    "                         MultiLayerFC(in_tensor=self._get_input('item_vfeature', train=train), \n",
    "                                      dims=self._dims, l2_reg=self._l2_reg_mlp, \n",
    "                                      dropout_mid=self._dropout_rate, train=train,\n",
    "                                      scope='item_MLP', reuse=not train),\n",
    "                         train=train)\n",
    "\n",
    "    def _build_default_fusions(self, train=True):\n",
    "\n",
    "        self._add_module('item_vec',\n",
    "                        Average(scope='item_average', reuse=not train, \n",
    "                                module_list=[self._get_module('item_vec', train=train), \n",
    "                                self._get_module('item_vf', train=train)], weight=2.0),\n",
    "                        train=train)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
